Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding find_with_linked #1728

Closed
wants to merge 9 commits into from
Closed

Conversation

darkmmon
Copy link
Contributor

PR Info

New Features

  • added find_with_linked

Bug Fixes

Breaking Changes

Changes

Comment on lines 121 to 133
pub(crate) fn new(query: SelectStatement) -> Self {
Self::new_without_prepare(query)
.prepare_select()
.prepare_order_by()
}

pub(crate) fn new_without_prepare(query: SelectStatement) -> Self {
Self {
query,
entity: PhantomData,
}
.prepare_select()
.prepare_order_by()
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we invoke prepare_order_by() method twice? When we call SelectTwoMany::new() it will invoke prepare_order_by() method twice.

@tyt2y3
Copy link
Member

tyt2y3 commented Jun 29, 2023

Just to clarify, the implementation should involve using a hash map, instead of relying on order by (which is theoretically wrong) to group elements.

The algorithm can be referenced in

let mut hashmap: HashMap<String, Vec<<R as EntityTrait>::Model>> =

Since now Value is already Hashable via SeaQL/sea-query#598, we should change the implementation of loader as well. Instead of HashMap<String, _> we should use HashMap<Value, _>

We should also consider chaning the existing find_with_related impl as well, but may be, not in the same PR.

Proceed with caution: I would rate this task ★★★★☆

@@ -997,6 +998,27 @@ where
L: EntityTrait,
R: EntityTrait,
{
// #[cfg(feature = "hashable-value")]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think sea_query::Value is not hashable by default and a specified feature is needed
but then the tests would need to also have the feature on in the configuration
my thought for solution is to make sea_query::Value to be hashable by default

Copy link
Member

@tyt2y3 tyt2y3 Jul 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the 'hashable-value' feature is intended to be added by default.

@darkmmon darkmmon marked this pull request as ready for review July 6, 2023 01:47
let last_val = last_l.get(col);
if !val.eq(&last_val) {
same_l = false;
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need a nested scope here?

}
acc

// let mut acc: Vec<(L::Model, Vec<R::Model>)> = Vec::new();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you want to keep the old code for reference, we can put it in another function consolidate_query_result_of_ordered_rows

src/executor/select.rs Outdated Show resolved Hide resolved
src/executor/select.rs Outdated Show resolved Hide resolved
src/executor/select.rs Outdated Show resolved Hide resolved
@tyt2y3
Copy link
Member

tyt2y3 commented Jul 6, 2023

@darkmmon I believe you liked performance algorithms, so may be we can optimize the implementation, here we want to minimize allocation, copy and (to some extend) move.

We observe that rows: Vec<(L::Model, Option<R::Model>)> is owned. We can do some operations in place, by mutating it. We can use https://doc.rust-lang.org/std/option/enum.Option.html#method.take to take the R::Model out from the vec, leaving a None in place. As such we don't have to re-allocate Vec<L::Model>.

So the intermediate hashmap can simply be L::PK -> Vec<R::Model>

In the final step, we can re-assemble everything together. Something like:

rows.into_iter().filter_map(|l_model, _| {
let l_pk = pk_of(l_model);
let r_models = hashmap.remove(l_pk);
if let Some(r_models) = r_models {
    Some((l_model, r_models))
} else {
    // it means we have seen this l_model already
    None
}
}).collect()

The key points here:

  1. L::Model should never be cloned, and only needs to be moved once.
  2. R::Model should never be cloned, and needs to be moved once: from input into the hashmap; the second move from the hashmap into the output is only a transfer of ownership of the Vec (which is just the pointer, not the blob of memory).

Anything more might be unnecessary. In Rust, objects are heavy as they are not pointers, so cloning and even moving them (to/from the heap) is somewhat expensive.

tyt2y3 pushed a commit that referenced this pull request Jul 7, 2023
@tyt2y3 tyt2y3 mentioned this pull request Jul 7, 2023
@tyt2y3
Copy link
Member

tyt2y3 commented Jul 7, 2023

GitHub somehow did not allow me to merge into a different branch. So I did manually.

@tyt2y3 tyt2y3 closed this Jul 7, 2023
@billy1624 billy1624 linked an issue Jul 10, 2023 that may be closed by this pull request
@github-actions
Copy link

github-actions bot commented Aug 2, 2023

🎉 Released In 0.12.1 🎉

Thank you everyone for the contribution!
This feature is now available in the latest release. Now is a good time to upgrade!
Your participation is what makes us unique; your adoption is what drives us forward.
You can support SeaQL 🌊 by starring our repos, sharing our libraries and becoming a sponsor ⭐.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consolidate find_with_* result with HashMap Missing method "find_with_linked"
3 participants